Image signal coding system

ABSTRACT

An adaptive blocking coding system selects an effective blocking of an input image signal to be encoded in accordance with the correlation between fields, even if motion is detected between the fields. The blocking patterns include an individual field blocking, a non-interlace blocking, a split blocking and an inverted split blocking. Further, the coding system searches for motion from both odd and even fields of a frame for producing a motion, compensated prediction signal in order to provide high-efficient coding.

This application is a divisional of application Ser. No. 08/803,235,filed on Feb. 20, 1997, now U.S. Pat. No. 5,867,220 which is acontinuation of application Ser. No. 08/121,293, filed Sep. 13, 1993,now U.S. Pat. No. 5,638,127, which is a divisional of application Ser.No. 07/962,299 filed Oct. 16, 1992, now U.S. Pat. No. 5,274,442, theentire contents of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION:

1. Field of the Invention

The present invention relates to an image coding system for coding animage signal with high efficiency.

2. Description of the Prior Art

As is known in the art, means for eliminating redundant componentsincluded in an image signal is used for coding an image signal. Atypical approach to image coding is the transform coding method whereinan image is divided into blocks, an orthogonal transform is carried outfor each of the blocks, and the transform coefficients are encoded.

In the case of television signals such as an NTSC signal, interlacedscanning is used whereby an image signal of one frame is scanned twice,once in the odd field and once in the even field. The two fields scandifferent but complementary spaces of an image. The fields have imageinformation at different times but there is a relatively strongcorrelation therebetween because the scanned lines of the two fields arealternate and adjacent. There is a technique in which coding is carriedout after combining the fields and dividing them into blocks when codingan image signal produced by the interlaced scanning.

FIG. 1 is a block diagram showing the structure of an embodiment of"High Efficiency Image Coding System" described in the Japanese PatentPublic Disclosure No. 1688/1991. In FIG. 1, the coding system includes anon-interlacing section 1, a motion detecting section 2, a non-interlaceblocking section 3, an individual field blocking section 4, anorthogonal transform section 5, a quantizing section 6 for quantizing aconversion coefficient at the output of the orthogonal transform section5, and coding section 7.

In operation, a series of input image signals 100, which are produced bythe interlaced scanning method and applied to each field, are convertedto a non-interlaced signal 101 in the non-interlacing section 1 asindicated in FIG. 2(C). As shown, the pixels belonging to the odd fieldand the pixels belonging to the even field appear alternately in everyother line.

When an object is stationary and the correlation between adjacent linesis high, it is effective to use a non-interlaced signal and to code theimage signal in a block including components from both fields. FIG. 3(A)shows an example of such a condition. On the other hand, when an objectis moving, the correlation between adjacent lines is lowered and it isconsidered to be effective to execute the coding in units of individualfields. This is because a non-interlaced signal is used for the movingobject results in discontinuation as shown in FIG. 3(B), causing a powerto be generated in high frequency coefficients during the transformcoding. In this case, the blocking as indicated in FIG. 3(C) isadequate.

Thus, the motion detector 2 detects the motion of an object and changesthe operation when the object is detected as being stationary by asignal 103 indicating motion, to conduct the blocking shown in FIG. 3(A)(hereinafter, this arrangement of FIG. 3(A) is called the non-interlaceblocking) in the non-interlace blocking circuit 3. If the object isdetected to be moving, the motion detector 2 changes the operation toconduct the blocking shown in FIG. 3(C) (hereinafter, this arrangementof FIG. 3(C) is called the individual field blocking) in the individualfield blocking circuit 4.

The blocks obtained by changing the blocking as explained above aresubjected to the discrete cosine transformation (DCT) in the orthogonaltransform section 5. The transform coefficients obtained as describedabove are quantized in the quantizing section 6, and a variable lengthcode is assigned in the coding section 7 in accordance with theoccurrence probability of respective events.

Since a conventional image coding system has been structured asdescribed above, it has been difficult to realize the blocking utilizingthe correlation between fields when an object is moving. Moreover, sucha system has not utilized the property of different intensities in powerdistribution of the coefficients after conversion caused by thedifference in arrangement of pixels within the block. In addition, thereis the difference in power between the stationary blocks and movingblocks. The moving blocks having a high signal power which has not beenutilized.

FIG. 4 is a block diagram of another conventional interframe predictivecoding system described, for example, in the transactions on the 3rdHDTV International Work Shop, "A Study on HDTV Signal Coding with MotionAdaptive Noise Reduction" (Vol 3, 1989). In FIG. 4, this systemcomprises a frame memory 21, a motion detecting section 22. a subtracter23, a coding section 24, a local decoding section 25, an adder 26 and amultiplexing section 27. Although omitted in this figure, the encodeddate is decoded at a receiving side in order to reproduce thetransmitted signal.

In operation, the motion of an object between the current field and thefield of the same type of the preceding frame is detected block byblock, the block consisting of a plurality of pixels of an input imagesignal 201 which is provided by the interlaced scanning method andformed of frames, each frame having both odd and even fields. The motionbetween odd fields is detected in the motion detecting section 22 bysearching the block which has the most distinctive resemblance to thecurrently processing block among the already encoded blocks 202,adjacent to the position corresponding to the currently processing blockin the odd fields stored within the frame memory 91. The degree ofresemblance is evaluated by using an absolute sum of differential valuesor a square sum of differential values of the corresponding pixels inboth blocks. The amount of motion in both horizontal and verticaldirections between the current block and the block determined to be themost similar is provided as a motion vector 203. The frame memory 21outputs a motion compensated prediction signal 204 corresponding to thismotion vector 203.

A prediction error signal 205 obtained in the subtracter 23 bysubtracting the motion compensated prediction signal 204 from the inputsignal 201 :is applied to the coding circuit 24 in which the spatialredundancy is removed. Since low frequency components of an image signalgenerally occupy a greater part of the power thereof information can becompressed by quantizing high power portions with a large number of bitsand quantizing low power portions with a small number of bits. Accordingto an example of this information compression method, the frequencyconversion is carried out for an 8×8 pixels block by conducting anorthogonal transform such as a discrete cosine transform toscalar-quantize the transform coefficients. The scalar-quantized codingdata 206 is sent to the local decoding section 25 and to themultiplexing section 27. The multiplexing section 27 conductsmultiplexing and encoding for the coding data 206 and the motion vector203 to output these signals to a transmission line 209.

Meanwhile, the local decoding circuit 25 executes the inverse operationof the operation in the coding section 24, namely the inverse scalarquantization and inverse orthogonal transform to obtain a decoded errorsignal 207. The motion compensated prediction signal 204 is added to thedecoded error signal 207 in the adder 26 and stored in the frame memory21 to detect motion of the odd field of the next frame.

In addition, the motion of the even fields of the input image signal 201with respect to the already encoded field of the frame memory 21 is alsodetected for the coding of the motion compensated prediction errorsignal. As described above, in the conventional interframe predictivecoding system, redundancy with respect to time included in moving imagesignals is removed by the motion compensated prediction coding andredundancy with respect to space is removed by the orthogonal transform.

Since the conventional interframe predictive coding system is structuredto individually encode both the odd field and even field by predictingthe current (present) odd field from the odd field of the alreadyencoded frame and predicting the current even field from the even fieldof the already encoded frame, the encoding efficieny is low because thespatial correlation existing between the continuous fields, produced bythe interlaced scanning method, is not used.

SUMMARY OF THE INVENTION

The present invention has been proposed to overcome the problems in theprior art. Therefore it is an object of the present invention not onlyto adaptively discriminate between a block which is effective fornon-interlace blocking and a block which is effective for individualfield blocking, but also to enhance coding efficiency by adding a classof blocking so that field correlation is used even for a moving image,the quantization accuracy is controlled and the scanning sequence oftransform coeffecients is changed in accordance with switching of theblocking.

It is another object of the present invention to provide a coding systemfor searching motion from both odd and even fields of the frame which isalready encoded in order to predict each present field.

It is a further object of the present invention to provide a codingsystem for enabling highly efficient coding by realizing blocking foradaptively switching the field and frame in the block coding ofprediction errors.

According to the first aspect of the present invention, an adaptiveblocking image coding system encodes an input image signal obtained byinterlaced scanning in a unit of the block of M pixels×N lines. Morespecifically, the adaptive blocking image coding system comprisesblocking means for selectively forming a first type block including onlythe pixels of M pixels×N lines belonging to the odd field of the inputimage signal or the pixels of M pixels×N lines belonging to the evenfield thereof, a second type block wherein the pixels of the Mpixels×N/2 lines belonging to the odd field and the pixels of the N/2lines belonging to the even field are arranged alternately in everyother line corresponding to scanning positions on a display screen, athird type block wherein the pixels of M pixels×N/2 lines belonging tothe odd field are arranged in the upper or lower half of the block andthe pixels of M pixels×N/2 lines belonging to the even field arearranged in the remaining half of the block, and a fourth type blockwherein the pixels of M pixels×N/2 lines belonging to the odd field arearranged in the upper or lower half of the block, the pixels of Mpixels×N/2 lines belonging to the even field are arranged in theremaining half of the block and the pixels of either field are invertedupside down in the vertical direction with respect to the displayscreen. The system further includes blocking determining means fordetermining the type of blocking by the blocking means, a transformmeans for orthogonally transforming the block formed by the blockingmeans, quantizing means for quantizing the transform coefficientobtained by the transform means, and coding means for encoding thequantized index obtained by the quantizing means.

With the structure described above, the block of the M pixels×N lines ifobtained by one blocking selected from the non-interlace blocking wherethe pixels of M pixels×N/2 lines belonging to the odd field and thepixels of M pixels×N/2 lines belonging to the even field are arranged inevery other line corresponding to the scanning positions on the displayscreen, an arrangement (hereinafter, called split blocking) where thepixels of M pixels×N/2 lines belonging to the odd number field arearranged in the upper half or lower half block and the pixels of Mpixels×N/2 lines belonging to the even field are arranged in theremaining half block, and an arrangement (hereinafter, called invertedsplit blocking) where the pixels of M pixels×N/2 lines belonging to theodd field are arranged in the upper or lower half block, the pixels of Mpixels×N/2 lines belonging to the even field are arranged in theremaining half block and the pixels of either field are inverted in thevertical direction with respect to the display screen. The obtainedblock is orthogonally transformed the transform coefficients arequantized, and then the quantization index is encoded.

According to the second aspect of the present invention, the quantizingmeans for quantizing the transform coefficient in the adaptive blockingimage coding system variably controls the quantization accuracy inaccordance with the type of arrangement blocking-processed by theblocking means.

More specifically, one of the arrangements including individual fieldblocking, non-interlace blocking, split blocking, or inverted splitblocking is orthogonally transformed, the transform coefficient isquantized and the quantizing index is encoded with the quantizingaccuracy in accordance with the information, indicating the selectedblocking.

According to the third aspect of the present invention, the coding meansfor encoding the quantizing index produced when quantizing the transformcoefficient in the adaptive blocking image coding system determines thescanning sequence (path), for quantizing the transform coefficient inaccordance with the type of arrangement to be blocking-processed by theblocking means.

More specifically, one of the arrangements to be blocking-processed bythe individual field blocking, non-interlace blocking, split blocking,or inverted split blocking is orthogonally transformed, the transformcoefficient is quantized, and the quantizing index is encoded with thequantization accuracy and the scanning sequence in accordance with theinformation indicating the selected blocking.

According to the fourth aspect of the present invention, the adaptiveblocking image coding system comprises blocking determining means forselecting the type of arrangement to be blocking-processed in accordancewith the value obtained by multiplying a predetermined weightingcoefficient with the pixels of each line included in the block and thentotaling such multiplied values.

More particularly, one of the arrangements to be blocking-processed byindividual field blocking, non-interlace blocking, split blocking, orinverted split blocking is selected by the value obtained by multiplyingthe predetermined weighting coefficient with the pixels of each lineincluded in the block and then totaling such multiplied values. Theselected block is orthogonally transformed, the transform coefficient isquantized and the quantizing index is encoded.

According to the fifth aspect of the present invention, the adaptiveblocking image coding system also comprises a blocking determining meansfor selecting the type of arrangement which has the minimum coefficientpower of a predetermined high frequency component among the transformcoefficients obtained by discrete cosine transform of the block.

In other words, one of the arrangements to be blocking-processed byindividual field blocking, non-interlace blocking, split blocking, orinverted split blocking is selected in such a manner that thecoefficient power of the predetermined high frequency element componentis the minimum among the transform coefficients obtained by discretecosine transform of the block. The determined block is orthogonallytransformed, the transform coefficient is quantized, and the quantizingindex is encoded.

According to the sixth aspect of the present invention, there isprovided a coding system which individually searches the motion fromboth odd and even fields of the already encoded frame in order topredict the field to be encoded, the system comprising the followingelements:

(a) input means for inputting an input signal to be encoded;

(b) a field memory for storing signals based on the input signal bydividing it into a plurality of fields such as the odd field and evenfield;

(c) predictive signal output means for outputting predictive signals ofa plurality of types predicting the change of input signal on the basisof signal stored in the field memory;

(d) a selector for selecting a predictive signal from the predictivesignals provided by the predictive signal output means; and

(e) coding means for encoding the input signal using the relationshipbetween the predictive signal selected by the selector and the inputsignal from the input means.

With such an arrangement, the coding system can provide stabilizedprediction efficiency regardless of motion of an object by makingreference to both fields of the already encoded frame for the purpose ofprediction.

According to the seventh aspect of the present invention, the codingsystem is structured to realize adaptive prediction from the searchedtwo kinds of motion compensated predictive signals and a plurality ofpredictive signals combining interpolation signals of these motioncompensated predictive signals.

Since the coding system as constructed utilizes a predictive signalproduced by interpolating the predictive signals from both fields of thealready encoded frame motion at the intermediate point of time and spaceof the two fields used for the prediction can be considered. Moreover,this coding system also functions as a low-pass filter, whereby theprediction efficiency can be improved and the encoded image isstabilized.

According to the eighth aspect of the present invention, the codingsystem executes the encoding, for example encoding prediction errorsignals, by adaptively switching the encoding operation from blocking ofthe pixels of only the odd field or even field of the frame forencodement to blocking of both odd and even fields for encodement, thesystem comprising the following elements:

(a) input means for inputting an input signal to be encoded by dividinginto a plurality of fields such as an odd field and even field;

(b) a blocking selection section for selecting, at the time of blockingand encoding the signal from the input means, a block suitable for theencoding between the block consisting of the signal of only one kind offield and the block consisting of the signal combining signals of aplurality of fields;

(c) a block forming section for forming a block selected by the blockingselection section; and

(d) coding means for encoding a block formed by the block formingsection.

The coding system having such a structure provides high efficiencyencoding by selecting the blocking method most suitable for theencoding, i.e., blocking the pixels of only either of the odd field oreven field, or blocking the pixels of both odd and even fields.

According to the ninth aspect of the present invention, the codingsystem also comprises a concrete selecting means for adaptivelyswitching the block selection. This selecting means includes any one ofthe following selecting mans:

(a) selecting means for selecting the block with the least amount ofencoding information from a plurality kinds of block;

(b) selecting means for selecting the block with the least amount ofencoding errors from a plurality kinds of block; and

(c) selecting means for selecting the block with the least amount ofhigh-frequency components in the signal to be encoded from a pluralitykinds of block.

The coding system having such a structure enables adaptive switching ofthe blocking by selecting the blocking with less encoding information,the blocking with less encoding errors, or the blocking with less highfrequency components included in the signal to be encoded, from theblocking of the pixels of one of only the odd or even field or theblocking of the pixels of both odd and even fields.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be more fully understood from the following detaileddescription and the accompanying drawings in which:

FIG. 1 is a block diagram of an image coding system in the prior art;

FIGS. 2(A) through 2(C) are diagrams for explaining non-interlaceblocking;

FIGS. 3(A) through 3(C) are diagrams for explaining an adaptive blockingof the prior art;

FIG. 4 is a block diagram showing the structure of another coding systemof the prior art;

FIG. 5 is a block diagram of an embodiment of the present invention;

FIG. 6 and FIGS. 6(A) through 6(D) are diagrams for explaining adaptiveblocking in the embodiment shown in FIG. 5;

FIG. 7 is a block diagram showing the structure of an adaptivefield/frame coding system of another embodiment of the presentinvention;

FIG. 8 is a diagram showing an exemplary input image signal;

FIG. 9 is a block diagram showing an example of the structure of aninterpolating section shown in FIG. 7;

FIGS. 10(A) and 10(B) are diagrams for explaining the operation of amotion detecting circuit;

FIG. 11 is a diagram for explaining the operation for using a motioncompensated predictive signal in the embodiment shown in FIG. 7;

FIG. 12 is a block diagram showing the structure of an adaptivefield/frame coding system according to another embodiment of the presentinvention;

FIG. 13 is a block diagram showing another example or the interpolatingsection;

FIG. 14 is a block diagram showing an adaptive field/frame coding systemaccording to embodiment of the present invention;

FIG. 15 is a block diagram showing an example of the structure of theblocking selection section;

FIG. 16 and FIGS. 16(A) through 16(C) are diagrams showing a structuralexample of the block selected by the blocking selecting section;

FIG. 17 is a block diagram showing a structural example of the blockingforming section;

FIG. 18 is a block diagram showing a structural example of the blockingdecomposing section;

FIG. 19 is a block diagram showing another structural example of theblocking selecting section;

FIG. 20 is a block diagram showing another structural example of theblocking selecting section;

FIG. 21 is a block diagram showing a structural example of the frequencyanalyzing section;

FIG. 22 is a diagram showing an example of the accumulated frequencycomponents; and

FIG. 23 is a block diagram showing another structural example of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to FIG. 5, an embodiment of the present invention is shown asan adaptive blocking image coding system. In FIG. 5, the image codingsystem comprises a non-interlacing section 1 for conductingnon-interlace processing; a blocking determination section 8; anindividual field blocking section 4; a non-interlace blocking section 3;a split blocking section 9; an inverted split blocking section 10; anorthogonal transform section 11; a quantizing section 12 and codingsection 13. Such various types of blocking are shown in FIG. 6. FIGS.6A-6D show individual field blocking non-interlace blocking, splitblocking and inverted split blocking, respectively.

The operation will be explained with reference to FIG. 5 and FIGS.6A-6D. The input image signal series 100 which is scanned by theinterlace scanning method and is inputted field by field is convertedinto a non-interlaced signal 101 in the non-interlace section 1.

FIG. 2 shows a profile of non-interlace processing in the prior artsimilar to the non-interlace processing in the present invention. When(A) is defined as an input image signal from the odd field and (B) as aninput image signal of the even field, the non-interlaced signal 101shown in (C), alternately combining the lines from respective fields canbe obtained.

The individual field blocking section 4 executes, as shown in FIG. 6(A),blocking in which the fields are processed individually. This blockingis effective when the correlation between the fields is not availablebecause of quick motion.

The non-interlace blocking section 3 executes the blocking shown in FIG.6(B). In the case of a stationary or still image, a continuous image canbe obtained by non-interlaced processing of the fields. The wavelengthof the signal thereby becomes substantially longer, resulting in powerbeing concentrated on low frequency components in the successivetransform coding.

The split blocking section 9 conducts the blocking as shown in FIG.6(C). This blocking is effective in the case where the correlationbetween the fields exists but the fields are noncontinuous whennon-interlace blocking is carried out.

The inverted split blocking section 10 conducts the blocking shown inFIG. 6(D). This blocking is also effective in the case where thecorrelation between fields exists but the fields are noncontinuous whennon-interlace blocking is carried out. This blocking preventsdiscontinuation at the center of the block when the split blockingmethod is used.

The blocking determining section 8 determines the optimum blocking froma plurality of blockings as explained above and outputs a blockingarrangement selecting signal 102 for selecting the determined blocking.Here, it is important to enhance the concentration of power, on the lowfrequency coefficients in the transform coding. For this purpose, it iseffective to evaluate the amplitude of high frequency components in eachblocking and select the blocking hasting the minimum amplitude.

In one of the evaluation methods, a weight is multiplied with the pixelsof each line and the obtained values are then totaled. For example, theweight of +1 is given to the lines 0, 2, 4, 6 using the line numbersshown in FIG. 6, and the weight of -1 is given to the lines 1, 3, 5, 7.Thereafter, the obtained values are totaled to obtain the absolute valueof the sum. Moreover, the weight +1 is given to the lines 8, 10, 12, 14,and the weight -1 is given to the lines 9, 11, 13, 15. The obtainedvalues are then totaled to also obtain the absolute values of the sum.Both absolute values are totaled. Thus, the weighting is invertedalternately for respective lines and it is equivalent to the evaluationof the maximum frequency component when non-interlace blocking has beenconducted.

Further, the weight +1 is given to the lines 0, 4, 8, 12 and the weight-1 to the lines 2, 6, 10, 14. The obtained values are totaled to obtainthe absolute value of the sum. In addition, the weight +1 is given tothe lines 1, 5, 9, 13 and the weight -1 to the lines 3, 7, 11, 15. Theobtained values are then totaled to obtain the absolute value of thesum. These absolute values are also totaled to evaluate the maximumfrequency component of the individual field blocking.

In addition, the weight +1 is given to the lines 0, 4, 1, 5 and theweight -1 to the lines 2, 6, 3, 7. The obtained values are totaled toobtain the absolute value of the sum. The weight +1 is also given to thelines 8, 12, 9, 13 and the weight -1 to the lines 10, 14, 11, 15. Theobtained values are totaled to obtain the absolute value of the sum.Both absolute values are then totaled to evaluate the maximum frequencycomponent of the split blocking.

The weight +1 is given to the lines 0, 4, 7, 3 and the weight -1 to thelines 2, 6, 5, 1. The obtained values are totaled to obtain the absolutevalue of the sum. Moreover, the weight +1 is given to the lines 8, 12,15, 11 and the weight -1 to the lines 10, 14, 13, 9. The obtained valuesare totaled to obtain the absolute value of the sum. These absolutevalues are totaled to evaluate the maximum frequency component of theinverted split blocking.

In another method for evaluation of each blocking, the number oforthogonal transformed coefficients having an amplitude larger than apredetermined threshold value for the respective blockings is counted,and the blocking having the minimum number is selected.

The orthogonal transform section 11 carried out the orthogonal transformof the selected block to obtain the transform coefficients. The obtainedtransform coefficients are quantized in a fixed sequence by thequantizing section 12. In this case, some difference lies in the powerof the coefficients depending on the type of blocking. In general,non-interlace blocking tends to be selected for a stationary region andthe power is comparatively small.

Meanwhile, since the correlation between fields becomes small in a quickmotion area, individual field blocking is often selected and the poweris large. Moreover, split blocking and inverted split blocking areconsidered to be intermediate to the above two blockings. Therefore,efficiency can be improved by variably controlling quantization accuracyin accordance with the type of blocking.

The quantizing accuracy can also be controlled variably in accordancewith not only the type of blocking but also the combination of actualsignal power and quantization error power. In this case, it is alsopossible to execute variable length coding by combining the informationindicating type of blocking and the information indicating quantizationaccuracy.

Indexes obtained by quantizing the coefficients are encoded in thecoding section 13. In this case, the coefficients are scanned from thosehaving a larger coefficient power to those having a smaller one in orderto enhance the efficiency of encoding. For the coefficients having apower lower than a certain specified value, the encoding may cease.Therefore, it is very convenient if the power distribution can beanticipated. There is a tendency with respect to the distribution ofpower of the coefficients that the power is increased as the frequencyis lower. However, if the blocking is adaptively changing, as in thepresent invention, the coefficients having lower power do not alwayscorrespond to low frequency components. Then, the coding efficiency canbe improved by changing the scanning sequence or path in accordance withthe type of blocking.

Since the present invention is structured as explained above, the,following effects can be obtained.

The coding efficiency of transform coding is improved by switching theblocking of an image signal scanned by the interlaced scanning methodinto an adapted blocking. Moreover, the efficient assignment ofinformation quantity can be realized by variably controlling thequantizing accuracy of transform coefficients correspondingly to theswitching of the blocking. In addition, the encoding efficiency can alsobe improved in transform coding by changing the scanning sequence of thetransform coefficients within the block.

Referring now to FIG. 7, a structural diagram of an adaptive field/framecoding system according to another embodiment of the present inventionis shown. The system includes an odd field memory 28 for storing localdecoded signals of odd fields, an even field memory 29 for storing localdecoded signals of even fields, an interpolation section 20 forinterpolating a predictive signal with motion compensated from the twofields, and a selector 21 for selecting a predictive signal which givesthe optimum prediction from three signals of the signals predicted fromthe odd and even fields and the interpolated predictive signal. In FIG.7, sections 200, 300 and 500 enclosed by a broken line respectivelydenote motion detecting means, predicting error signal output means andcoding means.

FIG. 8 shows a profile of input image signals 201 which are scanned bythe interlaced scanning method, wherein the odd and even fields arealternately applied. FIG. 8 shows the fields in the coordinates wheretime is plotted on the horizontal axis and vertical direction on thevertical axis. In FIG. 8, K1 indicates an odd field of the first frame,while G1, an even field of the first frame. In the same manner, K2 is anodd field of the second frame, while G2, an even field of the secondframe.

FIG. 9 is a block diagram of an example of the interpolating section 20.A simple arithmetic mean of the motion compensated prediction signal204a from the inputted odd fields and the motion compensated predictionsignal 204b from the inputted even fields is obtained and is used as aninterpolation predictive signal 204c.

The operation will be explained with reference to FIGS. 7, 8 and 9.Motion of the odd fields and even fields of the present frame inrelation to the preceding frame is detected in units of blocks includingpixels (n×m) in response to the input image signal 201 which is scannedby the interlace scanning method and includes the odd and even fieldsalternately. The motion of the odd fields between the present and thepreceding frames is detected by searching, in the motion detectingsection 22, the block which most resembles the currently processed blockin the image signal 201 from the blocks adjacent 202a to the positioncorresponding to the currently encoded object in the already encoded oddfields stored within the odd field memory 28.

As shown in FIG. 10, for example, it is assumed that image H1 existswithin one block unit (n×m) in the preceding frame, and the image movesto position H2 from position H1 in the present input image signal. Themotion detecting section 22 outputs a motion vector 203 which indicatesthe block has moved horizontally to H2 from H1. In this case, sincemotion is not detected in the vertical direction, the motion vector 203has the value of 0 with regard to vertical direction. The motion in thehorizontal and vertical directions thus obtained is outputted as themotion vector 203.

The odd field memory 28 outputs a motion compensated prediction signal204a corresponding to this motion vector 203. Similarly, compensationfor motion of the even fields in the preceding frame is carried out inthe motion detecting section 22, by searching the block resembling thecurrently processed block from the adjacent blocks 202b within the evenfield memory 29 and outputting the result as the motion vector 203. Themotion compensated prediction signal 204b corresponding to this motionvector 203 is outputted from the even field memory 29.

The interpolation processing is carried out in the interpolating section20 shown in FIG. 9, by using the motion compensated prediction signals204a and 204b to generate the interpolation predictive signal 204c,signal 204a being generated by motion compensated in accordance with themotion vector 203 and provided from the odd field memory 28, and motioncompensated predictive signal 204b being generated by motion compensatedin accordance with the motion vector 203 and provided from the secondfield memory 9. A predictive signal having the minimum error signalpower with respect to the currently encoding object block of the inputimage signal 201 is selected by the selector 21 from among the motioncompensated prediction signal 204a obtained from the odd field, themotion compensated prediction signal 204b obtained from the even field,and the interpolated motion compensated prediction signal 204c, and thenthe predictive signal 210 is produced.

FIG. 11 is a diagram showing the operation explained above. It isassumed that the odd field memory 28 shown in FIG. 7 stores an odd fieldK1 of the preceding (previous) frame, while the even field memory 29 ofFIG. 7 stores an even field G1 of the preceding frame. Here, the casewhere an odd field K2 and an even field G2 are included in the current(present) frame of the input image signal 201 will be discussed. First,when the odd field K2 is inputted, the motion compensated predictionsignal 204a from the odd field K1 of the preceding frame stored in theodd field memory 28 is provided to the selector 21. In the same manner,the even field G1 of the preceding frame stored in the even field memory29 is provided to the selector 21 as the motion compensated predictionsignal 204b. Then, the data of K1 and G1 are applied to theinterpolating section 20 and the interpolation processing as shown inFIG. 9 is conducted. Thereafter, such data is supplied to the selector21 as the motion compensated prediction signal 204c. The selector 21compares these three kinds of motion compensated prediction signals204a, 204b, 204c and the input image signal 201 to select the predictionsignal which has the minimum error signal power.

In the same manner, the selector 21 is responsive to the even field G2of the current frame to receive the prediction signal 204a based on theodd field K1 stored in the odd field memory 28, the motion compensatedprediction signal 204b based on the even field G1 stored in the evenfield memory 29, and the motion compensated prediction signal 204cobtained by the interpolation process on the basis of these motioncompensated prediction signals 204a, 204b based on both fields, and toselect the prediction signal which has the minimum error signal power.

In this embodiment (FIG. 7), the interpolation section is provided toconduct the interpolation processing based on the motion compensatedprediction signals 204a, 204b from the odd field memory 28 and evenfield memory 29 and thereby motion compensated prediction signal 204c isproduced. However, it is also possible that the interpolation section 20is not used as shown in FIG. 12. In this case, the motion compensatedprediction signal is generated in the selector 21 on the basis of thepreceding odd field K1 stored in the odd field memory 28 and thepreceding even field G1 stored in the even field memory 29 and theselector 21 selects the prediction signal minimizing the error signalpower in these two kinds of motion compensated prediction signals 204a,204b.

Further, in the embodiment shown in FIG. 7 the simple arithmetic meanhas been used for the interpolation section, but coding ensuring higherprediction efficiency can be realized by utilizing a weighted arithmeticmean taking into consideration field distance, as will be explainedhereunder with reference to FIG. 13.

FIG. 13 is a block diagram of an example of the interpolation circuit20. The motion compensated prediction signal 204a from the odd field ismultiplied by a weight α based on the distance to the field to beencoded, and the motion compensated prediction signal 204b from the evenfield is multiplied by a weight β based on the distance to the field tobe encoded. Thereafter, the arithmetic mean of these values is obtainedand the output thereof is used as interpolation predictive signal 204c.

The practical value of the weighting by the interpolation section 20 inrelation to the embodiment shown in FIG. 13 will be explained withreference to FIG. 11.

As shown in FIG. 11, when T is considered a unit of time for inputtingan odd field or an even field, there is a time difference of 2T betweenodd field K1 and odd field K2. On the other hand, there is a timedifference of T between even field G1 and odd field K2. Thus, theweights α and β can be determined by utilizing such time differences.For example, since the odd field K1 has a time distance of 2 T, theweight α is set to 1. Also, since even field G1 has a time distance of Tfrom odd field K2, the value of weight can be increased for the fieldhaving the lesser time distance by setting the value of β B to 2. In thesame manner, odd field K1 has a time distance of 3 T from even field G2and even field G1 has a time difference of 2 T. Thus, it is possible togive the value of weight which is proportional to the time difference bysetting α to 2 and β to 3 for weighting even field G2.

In the embodiment shown in FIG. 13, the weights α and β are determinedin the interpolating section on the basis of time distance. However, itis also possible that the weight α to be given to the odd field isalways set, for example, larger or smaller than weight β to be given tothe even field regardless of the time distance. Further, in thisembodiment, weights α and β used for the odd fields are different fromthose used for the even fields, but the weights for the odd fields maybe equal to those for the even fields. In addition, in this embodimentonly weights α and β are used, but the weights may be determined inaccordance with the other coefficients, for example, a coefficienthaving a quadratic function or another function having particularcharacteristics. Moreover, weights α and β do not have to be restrictedonly to one kind of value; it is possible that several kinds of weightsα and β are prepared and selected in accordance with the kind of inputsignal or the characteristic of input signal.

Another embodiment of the present invention will be explained withreference to FIG. 14.

The embodiment shown in FIG. 14 comprises a blocking selection section82 for selecting between an individual blocking of a prediction errorsignal for the odd and even fields and a non-interlace blockingincluding both odd and even fields; a blocking forming section 83 forconducting the blocking in accordance with the output of the blockingselection section 82; and a blocking decomposing section 84 fordecomposing the blocking to form the original field in accordance withthe block selection output. Section 400 enclosed by a broken linedenotes blocking means and the other sections 200, 300, 500 are similarto those shown in FIG. 7.

FIG. 15 is a block diagram of an example of the blocking selectionsection 82. The prediction error signal 205 is stored in the odd fieldmemory 31 for the odd field and in the even field memory 32 for the evenfield. As shown in FIG. 16(a) and 16(b), a block of p=16, q=16 isconsidered. The individual field blocking section 33 executes theblocking including the pixels of either of the odd or even field withinthe block of (p pixels×q lines), and these pixels are encoded in acoding section 35. As shown in FIG. 16(c), a non-interlace blockingsection 34 executes the blocking of (p pixels×q lines) included in theblock by alternately arranging the pixels of both odd and even fields,and these pixels are encoded in a coding circuit 36. The informationquantity comparing section 37 compares the quantity of data encoded inthe coding section 35 and the coding circuit 36, and outputs a blockingselection signal 211 indicating the blocking having the least amount ofinformation.

FIG. 17 is a block diagram of an example of the blocking forming section83. The prediction error signal 205 is stored in the odd field memory 41for the odd field and in the even field memory 42 for the even field. Inaccordance with the blocking selection signal 211 supplied from theblocking selection section 82, the blocking forming section 43 selectsthe blocking of the prediction error signals stored in the odd fieldmemory 41 and even field memory 42 from the blocking including pixels ofeither of the odd or even field within the block of (p pixels×q lines)and the blocking including pixels of both odd and even fields within theblock of (p pixels×q lines), and then outputs the blocked predictionerror signal.

FIG. 18 is a block diagram of an example of the blocking decomposingsection 84. The data decoded by a local decoding circuit 25 is appliedto the blocking decomposing section 44 in which the blocking isdecomposed in accordance with the blocking selection signal 211 from theblocking selecting section 82, and the decomposed block is then storedin the individual field memories 45, 46. The stored data is supplied asa decoded error signal 207.

The operation of this embodiment is explained hereunder.

The prediction error signal 205 obtained by subtracting the predictionsignal 210 from an input signal 201 in a difference circuit 23 is sentto the blocking forming section 83 shown in FIG. 17 and to the blockingselection section S8 shown in FIG. 15. The blocking selection section 82produces the blocking selection signal 211 for selecting the blockingincluding the pixels of either the odd or e-en field in the block of (ppixels×q lines), or the blocking including the pixels of both odd andeven fields in the block of (p pixels×q lines). The blocking formingsection 83 conducts individual field blocking or non-interlace blockingin units of (p×q) blocks in accordance with the blocking selectionsignal 211. The blocked signal Is applied to the coding circuit 24. Thecoding section 24 execute the orthogonal transform and sends the encodeddata 206 which is a scalar-quantized transform coefficient to both thelocal decoding section 25 and the multiplexing section 28.

After the inverse scalar-quantization and inverse orthogonal transformby the local decoding section 25, the data is decomposed into the oddand even fields in the blocking decomposing section shown in FIG. 18which decomposes the blocking into the fields in accordance with theblocking selection signal 211 in order to obtain the decoded differencesignal 207. The local decoded signal 208 obtained by adding a predictivesignal 210 to the decoded difference signal 207 in the adder 207 isstored in the first field memory 28 when it is the odd field or in thesecond field memory 29 when it is the even field, to detect the motionof each field of the next frame.

In this embodiment, a unit of blocks is formed of p=16, q=16, but it isdesirable that the values of p and q have the following relationshipwith the block size n×m used by the motion detecting section 22 asexplained in the embodiment shown in FIG. 7:

p=n, q=2m

Since DCT transform is often carried out in the block unit of 8 pixels×8lines, the size of 16 pixels×16 lines combining four block units isselected as the values of p and q in the blocking forming section. Inthis example, since P=n, n=16 pixels. Also, since q=2m, m=8. Thus, it isdesirable that the number of lines be reduced to 8 because the motiondetecting section 22 detects motion for both the odd and even fields.Meanwhile, since it is possible to employ the blocking combining the oddfield and even field in the blocking forming section, it is desirable toform a block of 16 lines including the odd and even fields.

In the embodiment shown in FIG. 14, the blocking has been selected bycomparing the quantity of information generated as shown in FIG. 15, butcoding based on the quality of encoding can be realized by selecting theblocking on the basis of the comparison of encoding quality as shown inFIG. 19.

FIG. 19 is a block diagram of an example of the blocking selectionsection 82. The predicting error signal 205 is stored in the odd fieldmemory 51 for the odd field and in the even field memory 52 for the evenfield. The individual field blocking section 53 realizes the blockingincluding the pixels of either the odd field or the even field withinthe block of (p pixels×q lines), and the coding/decoding section 55enables encoding/decoding. At the same time, the non-interlace blockingsection 54 realizes the blocking including the pixels of both fieldswithin the block of (p pixels×q lines), and the coding/decoding circuit56 enables coding/decoding. The difference between the encoded/decodeddata of the individual field blocking and the data just before theencoding is compared with the difference between the encoded/decodeddata of the combined field blocking and the data just before theencoding, by the error comparator 59 in order to select the blockingwith less errors and to provide an output as the blocking selectionsignal 211.

In the embodiment shown in FIG. 14, the quantity of generatedinformation has been compared for the selection of the block, while inthe embodiment shown in FIG. 19, the encoding errors have been compared.However, encoding with higher efficiency can be realized when conductingencoding utilizing the orthogonal transform, by selecting the blockingon the basis of the comparison of frequency components produced by thedifference of blocking as shown in FIG. 20.

FIG. 20 is a block diagram of an example of the blocking selectioncircuit 82. The predicting error signal 205 is stored in the odd fieldmemory 61 for the odd field and in the even field memory 62 for the evenfield. The individual field blocking section 63 executes the blockingincluding the pixels of only either the odd field or even field withinthe block of (p pixels×q lines), and a frequency analyzing section 65such as that shown in FIG. 21 executes the frequency analysis. Thenon-interlace blocking circuit 64 executes the blocking including pixelsof both fields within the block of (p pixels×q lines), and a frequencyanalyzing circuit 66 such as that shown in FIG. 21 executes thefrequency analysis. The blocking with fewer high-frequency components isselected from the individual field blocking and the combined fieldblocking to output the blocking selection signal 211.

FIG. 21 is a block diagram of an example of the frequency analyzingsections 65 and 66. The signal obtained by individually blocking the oddand even fields from the individual field blocking circuit 63, and thesignal obtained by blocking the pixels of both odd and even fields fromthe non-interlace blocking section 64, are supplied to sections 65 and66. These signals are converted to a signal in the frequency domain froma signal in the pixel domain using the orthogonal transform 68. Thehigh-frequency components are extracted from the converted signal in thefrequency domain by a high-frequency component selector 69 and theextracted high-frequency components are totaled by a high-frequencycomponent accumulator 70. The accumulated high-frequency components arecompared in a high-frequency component comparing section 67 to selectthe blocking with fewer amount high-frequency components.

FIG. 22 shows an example of the components accumulated by thehigh-frequency component adder 70 from the orthogonal transformedfrequency domain signal. Here, eight components, for example, having themaximum frequency component in the vertical frequency component, areselected.

In this embodiment, the coding section 24 does not use the selectioninformation of predictive signals or the selection information ofblocking, but according to another embodiment shown in FIG. 23, finercontrol is possible and high encoding quality can be realized byinputting an output of the selector 11 as the selection signal for thepredictive signal and the blocking selection signal as the selectionsignal for the blocking to the coding section 24 and by controlling theencoding characteristic with the selected prediction signal and theinformation of the selected blocking.

As explained above, the embodiment of FIG. 7 relates to a system forrealizing predictive coding of an input image signal obtained by theinterlaced scanning method with the motion compensation. The systemincludes motion detecting means for obtaining, for the odd or even fieldof the input image signal, the amount of displacement, in order to carryout the individual motion compensated prediction, in units of the blockof (n pixels×m lines) (n and m: positive integer) from both the odd andeven fields of the already encoded frame, and the prediction errorsignal output means for selecting, with a selector 21, the predictivesignal indicating the optimum prediction from signals including a firstpredictive signal 204a obtained by the motion compensation from the oddfield, a second predictive signal 204b obtained by the motioncompensation from the even field, and a third predictive signal 204cobtained by interpolating the first and second predictive signals inorder to obtain the difference from the field of the input signal andoutput the result as the prediction error signal.

Moreover, the embodiment of FIG. 7 is an adaptive field/frame codingsystem characterized in that the interpolation means for obtaining thethird predictive signal is the simple arithmetic mean of the firstpredictive signal and the second predictive signal.

Thus, the hardware can be minimized in size and encoding with higherprediction efficiency can be realized by generating an interpolationsignal of the predictive signal by simply obtaining the arithmetic meanof both predicted odd and even fields with motion compensation.

Further, the embodiment of FIG. 13 is an adaptive field/frame codingsystem characterized in that the interpolation means for obtaining thethird predictive signal is the weighted arithmetic mean of the firstpredictive signal and the second predictive signal, also considering thetime distance of the field used for the prediction and the field to beencoded.

Thus, encoding ensuring very high prediction efficiency can be realizedby generating the interpolation signal from the weighted arithmetic meanof both predicted odd and even fields with the motion compensation,while considering the time distance of the field used for the predictionand the field to be encoded.

The embodiment shown in FIG. 14 is an adaptive field/frame coding systemcomprising means for enabling encoding by selecting blocking includingthe pixels of either the odd field or even field within the block of (ppixels×q lines), or blocking including the pixels of both odd and evenfields within the block of (p pixels×q lines), in order to encode theprediction error signal for the odd and even fields of the input imagesignal in units of the block of (p pixels×q lines) (p and q: positiveinteger).

Moreover, the embodiment shown in FIG. 14 is an adaptive field/framecoding system characterized in that the blocking means for enablingencoding while selecting the blocks comprises selecting means forselecting the blocking with less information for encoding from blockingincluding the pixels of only one of the odd field and even field withinthe block of (p pixels×q lines), and blocking including the pixels ofboth odd and even fields within the block of (p pixels×q lines).

The embodiment shown in FIG. 19 is an adaptive field/frame coding systemcharacterized in that the blocking means for enabling encoding whileselecting the blocks comprises means for selecting the blocking withless encoding error from blocking including the pixels of only one ofthe odd field and even field within the block of (p pixels×q lines), andblocking including the pixels of both odd and even fields within theblock of (p pixels×q lines).

The embodiment shown in FIG. 20 is an adaptive field/frame coding systemcharacterized in that the blocking means for enabling encoding whileselecting the blocks comprises selecting means for selecting theblocking with less high-frequency components included in the signal tobe encoded from blocking including the pixels of only one of the oddfield and even field within the block of (p pixels×q lines), andblocking including the pixels of both odd and even fields within theblock of (p pixels×q lines).

In addition, the embodiment shown in FIG. 23 is an adaptive field/framecoding system characterized by enabling encoding while selecting thequantization characteristic of the transform coefficient in accordancewith the selected predictive signal and the selected blocking, in thecase of employing the orthogonal transformer and carrying out encodingby the quantization of transform coefficient in the coding section forthe encoding in units of the block of (p pixels×q lines).

In the above embodiments, an input image signal 201 is formed of theframe including the odd field and even field. However, the use of theodd field and even field is intended to show only an example, and thefield is not restricted to the odd or even field. The present inventioncan be useful whenever one frame is divided into fields, the odd fieldand even field being only examples of such fields of a frame. Forinstance, the present invention can also be applied to a case of storingdata by dividing the frame into two fields every two lines by, forexample defining the first field as the 1st and 2nd lines and the secondfield as the 3rd and 4th lines, and defining the first field as the 5thand 6th lines and the second field as the 7th line and 8th line, etc.Moreover, in addition to dividing a frame into two kinds of fields, suchas the odd field and the even field or the first field and the secondfield, the present invention can also be applied to the case of dividinga frame into more than two fields, for example, three or four kinds offields. In such a case, the number of field memories corresponds to thenumber of kinds of fields, and the processing explained above is carriedout for each field.

In the above embodiments, the blocking selection section selects theblocking from two kinds of blocking, including the blocking of thepixels of only one of the odd field and even field and the blocking ofthe pixels of both odd and even fields. However, the blocking mayinclude various combinations when two or more fields are prepared inaddition to the odd and even fields. The blocks shown in FIGS. 16(a),(b), (c) are only examples and various block forming methods may be usedto form the block other than the blocks of FIG. 16.

In the above embodiments, the blocking means shown in FIG. 14 is usedwith the prediction error signal output means and motion detectingmeans. Even if the sections other than the blocking means 400 arereplaced with conventional means, the 8th and 9th aspects explainedabove can be provided.

According to the 6th and 7th aspect explained above, a stable encodedimage with high efficiency can be obtained by individually searching themotion from each field of the already encoded frame to predict eachfield and by conducting adaptive prediction from the searched motioncompensated predictive signals (and interpolation signals).

In addition, according to the 8th and 9th aspects explained above, astable encoded image with high efficiency can also be obtained byadaptively selecting the encoding from the blocking of the pixels ofonly one of the fields of the frame to be encoded, and the encodingafter conducting the blocking of the pixels of the respective fieldswhen encoding the prediction error signal.

What is claimed is:
 1. A video signal encoder for encoding a firstmotion video signal representative of sequential video images includingfirst and second video images into a second motion video signalcomprising:a predictive error signal generator for generating apredictive error signal representative of the error in the second videoimage based on a prediction formed at least in part from the first videoimage; a coder for transform coding the predictive error signal toproduce a coded predictive error signal, said coder varying the scanningsequence of the transform coefficients thereof; a local decoderdetecting the scanning sequence of transform coefficients used intransform coding, and decoding the transform coefficients of the codedpredictive error signal to produce information representative of thefirst motion video signal including the first video image; a combiner,operatively connected to said local decoder, and combining thepredictive error signal with a predictive signal; a field memory,operatively connected to said combiner, for storing informationrepresentative of the first video image as plural image fields; apredictive signal generator, operatively connected to said field memory,and supplying the predictive signal from the plural fields stored insaid field memory to said predictive error signal generator and saidcombiner; wherein a signal representative of the second video image ofthe first motion video signal is assembled from the predictive errorsignal developed from a difference between the first video image and thesecond video image; wherein the coded predictive error signal from saidcoder is output as the second motion video signal.
 2. The encoder ofclaim 1 wherein said coder varies the scanning sequence used intransform coding based on encoding efficiency.
 3. The encoder of claim 1wherein:said predictive error signal generator includes a subtracter,operatively connected to said predictive signal generator, andsubtracting the predictive signal from a signal representative of thesecond video image of the first motion video signal to form thepredictive error signal; and a block former, operatively connected tosaid subtracter, and forming the predictive error signal into blocks;said coder being operatively connected to said block former, andencoding the blocked predictive error signal to form the codedpredictive error signal.
 4. A video signal conversion system forconverting between a first motion video signal representative ofsequential video images including first and second video images and asecond motion video signal comprising:a decoder detecting the scanningsequence of transform coefficients used in transform coding, anddecoding the transform coefficients to produce informationrepresentative of the first and second video images; a field memory forstoring information representative of the first video image as pluralimage fields; a predictive signal generator, operatively connected tosaid field memory, and supplying a predictive signal from the pluralfields stored in said field memory; wherein a signal representative ofthe second video image is assembled from a predictive error signaloutput from said decoder and developed from a difference between thefirst video image and the second video image; and a combiner,operatively connected to said field memory, and combining the signalrepresentative of the second video image with the predictive signalproduced by said predictive signal generator.
 5. The video signalconversion system of claim 4 wherein the scanning sequence used intransform coding the transform coefficients is determined based oncoding efficiency.
 6. The video signal conversion system of claim 4wherein the conversion system converts from the first motion videosignal to the second video signal.
 7. The video signal conversion systemof claim 4 wherein said field memory stores plural image fields and saidpredictive signal generator supplies plural predictive signals from theplural image fields, said predictive signal generator including,aninterpolator, operatively connected to said field memory, interpolatingat least some of the plurality of predictive signals and generating aninterpolated predictive signal which is different from any of theplurality of predictive signals supplied by said predictive signalgenerator, and a selector, receiving the plural predictive signals andthe interpolated predictive signal, and selecting a predictive signalfrom the plurality of predictive signals and the interpolated predictivesignal.
 8. The video signal conversion system of claim 7 wherein saidcombiner includes an adder, said adder adding the predictive errorsignal and the interpolated signal produced by said interpolator andsupplying the output thereof to said field memory.
 9. The video signalconversion system of claim 7 further comprising:a subtracter,operatively connected to said selector, and subtracting one of theplural predictive signals including the interpolative predictive signalfrom a signal representative of the second video image of the firstmotion video signal to form the predictive error signal; and an encoder,operatively connected to said subtracter, and encoding the predictiveerror signal to form an encoded predictive error signal, wherein saidencoder varies the scanning sequence used in transform coding thetransform coefficients; said decoder being operatively connected to saidencoder, for decoding the encoded predictive error signal for supply tosaid combiner.
 10. The video signal conversion system of claim 7 whereinthe interpolator produces the interpolated predictive signal bycomputing the arithmetic mean of at least some of the plural predictivesignals.